Quasi-Newton methods for machine learning: forget the past, just sample

We present two sampled quasi-Newton methods (sampled LBFGS and LSR1) for solving empirical risk minimization problems that arise in machine learning. Contrary to the classical variants of these sequentially build Hessian or inverse approximations as optimization progresses, our proposed sample points randomly around current iterate at every iteration produce approximations. As a result, constructed make use more reliable (recent local) information do not depend on past could be significantly stale. Our algorithms are efficient terms accessed data (epochs) have enough concurrency take advantage parallel/distributed computing environments. provide convergence guarantees methods. Numerical tests toy classification problem well popular benchmarking binary neural network training tasks reveal outperform their variants.